1. Field of the Invention
The present invention relates to image processing systems and, more particularly, to an image cytometer which performs image separation on a specimen.
2. Description of the Prior Art
An automated system for analysis of anchorage-dependent cells may contribute to improved understanding of many cellular functions. In vitro, anchorage-dependence is defined by the requirement that cells be attached to a substrate to proliferate. Many cell types exhibit anchorage-dependence and the loss of this requirement for attachment is usually associated with malignant transformation. (See, e.g., R. I. Freshney, "The Transformed Phenotype," in Culture of Animal Cells, a Manual of Basic Technique, 2nd ed., New York: Alan R. Liss, pp. 197-206, 1987.) In vivo, cell division, cell shape, cell migration, and control of cell growth and differentiation are all at least altered by interaction of cells with a substrate. For a fully automated cytometer to have the potential to analyze cellular parameters that depend on contact of cells with a substrate, the system would have to perform analyses in situ. The basis for such a system is analysis of images of the cells acquired from a computer-controlled microscope. The images acquired by scanning a specimen of cells would be analyzed to perform cytometry, the measurement of individual cells. The instrument that performs this task, the scanning or image cytometer, would ideally indicate: 1) the size and shape of cells, nuclei and key organelles; 2) the distributions and concentrations of important cellular substances; and 3) the organizational relationships of cells. Such an instrument should scan a specimen rapidly, nondestructively, and repeatedly to analyze the dynamics of large numbers of cells.
There are many potential applications for scanning or image cytometry. Cell division, for example, could be analyzed directly by repeatedly scanning a large population of cells as they progress through the phases of the cell division cycle. Current methods for measuring durations of the cell cycle phases include bromodeoxyuridine pulse labeling in flow cytometry and time-lapse cinematography in microscopy. Flow cytometry permits rapid analysis of large numbers of cells, but cannot repeatedly measure a given cell, and time-lapse cinematography allows repeated measurements of the same cells, but cannot be used for more than one or a few cells.
The scanning cytometer should bridge this gap, making it possible to track the lineage and cell cycle phase times of each cell in a large group, and differentiate between those that are quiescent and those that continue to divide. It should also be possible to correlate changes in nuclear size, shape and chromatin distribution with progression through the cell cycle. (See, e.g., C. De Le Torre and M. H. Navarrete, Exp. Cell Res. 88: 171-174, 1974; W. Sawicki, J. Rowinski and R. Swenson, J. Cell Physiol., vol. 84: 423-428, 1974; F. Giroud, Biol. Cell 44: 177-188, 1982.) In addition to improving understanding of cell division, certain scanning cytometry measurements may be sensitive indices of cell health. If so, scanning cytometry may lead to practical, automated assays for toxins, drugs and infective agents.
A number of techniques for cell measurement have been used with limited success. For example, one may use the "metabolic rate" method disclosed by E. A. Dawes, Quantitative Problems in Biochemistry, Baltimore: Williams and Wilkins, pp.293-311, 1972, or the "pooled quantity" method described in R. I. Freshney, "Quantitation and Experimental Design," in Culture of Animal Cells, a Manual of Basic Technique, 2nd ed., New York: Alan R. Liss, pp. 227-256, 1987. These two types of techniques are used to analyze the culture as a whole. Measurement of the metabolic rate of cells by CO.sub.2 production or O.sub.2 consumption allows analysis of live cells without disruption, but yields no individual cell data. There are also a number of methods for measuring amounts of substances from the whole culture that require cell destruction. Such chemical methods include the modified Lowry assay for proteins and DNA extraction assays.
In addition, there are automated or semi-automated devices available for cell measurement. In these devices, such as the Coulter counter and the flow cytometer (also known as the FACS or fluorescence activated cell sorter), measurements are typically made on individual cells in suspension. The Coulter counter provides the simplest individual cell data, i.e., cell number. More advanced electronic counters also measure cell size. However, the Coulter counter tends to give a count only, and has the disadvantage of causing cellular disruption. The flow cytometer gives cell number and size, as well as the quantity of cellular substances labeled with a specific fluorescent dye. The flow cytometer provides very low spatial resolution, however, and also tends to disrupt the cell culture under examination. Finally, while both the Coulter counter and the flow cytometer provide data on individual cells, they cannot analyze cells attached to a substrate. There are many advantages to measuring cells without disturbing them in their "native" location (e.g., attached to a substrate), and these advantages are not possible with the above-noted devices.
Measurements of individual cells in situ have been performed with varying degrees of automation in microscopy. The simplest approaches, such as utilization of a hemocytometer, require the microscopist to count cells and record information by hand. More complicated techniques, made possible by modifications to the microscope, allow quantitation of specific cellular parameters.
The first measurements of specifically labeled cellular substances were performed using a photomultiplier tube. Use of a photomultiplier tube requires positioning the cell of interest under an aperture and does not allow measurements of shape and size. The combination of image analysis and microscopy yields size, shape, and pattern measurements and allows quantitation of labeled cellular substances (see, e.g., B. H. Mayall, "Current Capabilities and Clinical Applications of Image Cytometry," Cytometry Supplement 3: 78-84, 1988). There are numerous advantages in the added information available from microscope images, but the methods used to analyze them are generally more labor intensive, resulting in the analysis of much smaller numbers of cells than with flow cytometers or electronic counters. Standard microscopic methods of cell measurement are also impaired, in that they require significant human interaction and are thus quite slow. In addition, such methods have the further disadvantage of producing data that is quite subjective and is based on a limited number of cells.
Complete automation of image cytometry is necessary for practical analysis of large numbers of fixed cells and efficient repeated scanning of groups of live cells. A summary of some of the systems capable of fully automated measurement of cellular specimens is given in Table I, below. The first of these systems counts cell colonies and measures colony size, but returns no individual cell data. Fully automated analysis of cell motility has been implemented for both single cells and groups of cells (see, e.g., G. Thurston, et al., "Cell Motility Measurements with an Automated Microscope System," Exp. Cell Res. 165: 380-390, 1986). In that report, location was recorded with each scan but no attempt was made to analyze cell size or shape, or the quantity of cellular substances.
Others have reported success at determining whether or not a smear contains malignant cells with instruments capable of rapidly scanning a microscope slide. The machine diagnosis was compared with the expert opinion of a pathologist. Presumably, the machine diagnosis was based on the shape and density of the cell nuclei. Data such as DNA content and nuclear size compiled from the individual cells, however, was not presented. It is, therefore, impossible to know whether the methods used for automated cytology might be adapted to allow precise measurements of cell shape and size, or the quantity of cellular substances.
TABLE I ______________________________________ Previous Image Cytometry Automation Application Recognition Measurements ______________________________________ Colony Counting computer, colony number, phase contrast size Cell Motility computer, location, phase contrast movement Cytology computer, malignant vs. fluorescence, nonmalignant PAP, Faulgen, etc. ______________________________________
Measurements of the cell nucleus and DNA content have been the focus of many cytometric studies because nuclear abnormalities are often associated with malignancy and because nuclear changes define the differences between the phases of the cell cycle. Other investigators have reported working with systems capable of various levels of automation for nuclear analysis. The SAMBA system has been used to measure DNA content on as many as 600 cells/experiment (see, e.g., E. Colomb, et al., Cytometry 10: 263-272, 1989) and the LEYTAS system has been used to measure DNA content on 100-300 cells/experiment (see, e.g., C. J. Cornelisse, et al., Cytometry 6: 471-477, 1985). The number of cells analyzed in these experiments is much smaller than the 10.sup.4 -10.sup.5 cells that can be analyzed in flow cytometry. A recent image cytometry review (see B. H. Mayall, Cytometry Supplement 3: 78-84, 1988), which presented DNA content experiments with 200 cells each, identified the necessity for operator interaction as a major impediment in the analysis of larger numbers of cells.
Image cytometry usually requires interactive selection of the objects of interest. During interactive operation a technician must either draw object borders with the aid of a digitization tablet or mouse, or utilize semi-automated techniques based on intensity thresholding and editing of incorrectly chosen objects. For example, the patent to Bacus (U.S. Pat. No. 5,018,209) discloses one such operator assisted system.
An image cytometry system that can perform nuclear analysis unattended by an operator has not yet been reported, prior to the present disclosure. Some review articles which provide a frame of reference for appreciating the improvements and novel features of the present invention are as follows: Roberts, J. NIH Research 2: 77 (1990); Herman, et al., Arch. Pathol. Lab. Meth. 111: 505 (1987); Baak, Path. Res. Pract. 182: 396 (1987); and Mayall, Cytometry Supplement 3: 78 (1988).
Accurate computer recognition of the cell nuclei in an image is the first step in fully automated measurement of DNA content and nuclear size, shape and pattern. In an image of cells stained with a fluorescent dye specific for DNA, computer recognition consists of correctly segmenting the image into bright foreground objects and dark background. There are many examples of image segmentation or cell edge finding techniques used for computerized recognition (see, e.g., L. O'Gorman, et al., IEEE Transactions on Biomedical Engineering 32: 696-706, 1985). It is difficult, however, to compare the performance of these different techniques because each method was developed for a specific application and demonstrated on only one or a few images. An assessment of the reliability of these techniques on large numbers of cells, with presentation of measurements such as DNA content, was not provided. The simplest of these methods, intensity thresholding, has been evaluated for measurement of the DNA content of fluorescent stained smears (see, e.g., T. Takamatsu, et al., Acta Histochem. Cytochem. 19: 61-71, 1986.). Thresholding resulted in lower precision than attained by flow cytometry. In that report, unreliable recognition was identified as a probable source of error.
In the field of image processing, image segmentation, i.e., the automated separation of objects from a background in a digital image, is a recurring theme. Previous methods for image segmentation, or object recognition, have included thresholding (or clustering), edge detection and region extraction (K. S. Fu and J. K. Mui, "A Survey on Image Segmentation", Pattern Recognition, vol. 13, pp. 3-16, 1981).
In thresholding, the computer utilizes differences in image intensity to delineate features from a background. In its simplest form, the image is thresholded into two intensity ranges. All pixels (or picture elements) below the threshold intensity value are separated into one group while all those equal to or above that value are separated into a second group. In more complicated methods, multiple thresholds are used and the threshold values are determined by a method called "clustering." Each set of intensity ranges can be used to identify a different type of object if intensity differences are well defined. Difficulties arise when the objects contain a broad range of intensities, or when object edges are characterized by gradual, rather than abrupt changes from internal intensity to background intensity.
In edge detection, the edges of the objects are assumed to occur where there are large changes in intensity within a short distance (small neighborhood of pixels) in the image. These steep intensity gradients can be enhanced by edge filters (convolution or Fourier). After the filter is applied, the edges appear either as white pixels on a black background or black pixels on a white background. Thresholding can then be used by a processor to locate the edge pixels. These edge pixels must then be connected by the processor and sorted to form separate boundary representations of each individual object in an image. The most complicated (and processor intensive) step involves connecting the edge pixels into continuous boundaries. In many objects, filtering results in disconnected and spotty edges that are difficult to connect and sometimes result in the joining of separate but close proximity objects. One such edge detection system was disclosed by Martin (U.S. Pat. No. 4,561,104).
Region extraction methods depend on searching sets of image pixels for similarities and grouping them according to predetermined criteria. This method sequentially searches arbitrary regions of the image for similarities or differences. If two adjacent regions are similar they are merged and if a single region is found to contain too much variability it is divided. Region merging and/or division is carried out repeatedly until the algorithm determines that the image has been segmented as well as possible. The difficulty with this method is in finding similarity criteria for grouping that can be easily implemented by computer. The other problem is that the repeating (iterative) nature of these methods can make them too slow for all but the fastest, most expensive computers.
In addition, image segmentation would be enhanced if an image cytometer had a stable source of light. Arc lamps are known to emit intensity that varies. Some of this time varying intensity is due to arc wander. Arc wander means that the source of the luminescence changes position with time. Arc wander causes the light cast on the microscope specimen to change intensity in a way that depends on the location within the microscope field of view. This means that one spot in the microscope field can increase in intensity while another spot simultaneously decreases in intensity. Due to this spatially dependent intensity variation it is not possible to place a light measuring device, such as a photodiode, at any one place in the light beam and utilize its signal to correct for intensity variation. It has been previously observed (G. W. Ellis, "Microscope Illuminator with Fiber Optic Source Integrator," J. Cell Biol. 101:83a, 1985) that placing an optical fiber in the microscope light path scrambles the light so that spatially dependent variation no longer exists.
Another problem with fluorescence microscopes, however, is the accumulation of enough light to cause a visible fluorescence in the specimen to be studied. Specially designed, aberration corrected optical elements are usually used to transfer illumination from an arc lamp to the microscope field. Because light transmission in optical fibers is not perfect, use of a fiber-optic light scrambler, such as an optical fiber in the light path of the microscope, further decreases the amount of light available for fluorescence excitation.
Light measurement in fluorescence image cytometry is also prone to error because current camera and image processor systems acquire and operate on 8-bit images. An 8-bit intensity measurement incorporates 256 discrete intensity values, 0 through 255. This is not enough to encompass the entire range of intensity found in an image of fluorescent-stained cell nuclei. In practice, some intensities in the brightest nuclei are actually greater than 255 and these values are incorrectly recorded as 255. This results in an underestimation of some of the integrated intensity measurements by the instrument. One way to correct this problem is to decrease the light intensity by a known amount and remeasure the portion of the microscope field that was too bright. This can now be done only by mechanically introducing a filter into the light path. The problem is that arc lamp intensity changes with temperature and a change in the electric current powering the arc lamp is followed by a slower change in temperature. This makes it essentially impossible to control arc lamp intensity by altering current alone.
Therefore, what is needed is a degree of automated image segmentation far greater than that achieved via use of all presently known image cytometers, including those described above. In particular, it would be desirable to analyze a significantly greater number of cells than is possible with other devices, requiring only minimal operator interaction at the very beginning of the procedure and data organization at the end, with complete operator independence during the actual measurement process. Furthermore, it would be an advantage if an image cytometer could stabilize and control the intensity of an arc lamp.